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BACKGROUND OF THE INVENTION 

[0001] The human psycho visual system is highly capable of appropriately adapting to 
illumination characteristics of an image. A human observer detects a particular range of colors 
for a large variety of scene illuminants. For example, the human viewer accurately perceives 
color of an object independent of the color of light illuminating the scene, hi contrast, color 
digital systems such as digital cameras, frequently infer the color of the scene illuminant 
incorrectly. 

[0002] Color appearance models can be used to model the manner the human visual 
system perceives the color of an object under different lighting conditions and with different 
backgrounds. The model is typically used so that an image viewed under one lighting condition 
can be adjusted to appear to have the same colors viewed under a different lighting condition. 

[0003] Color appearance models are intended to attain a capability to store images in a 
device-independent manner. For example, a computer of an imaging system can acquire an 
image and a color appearance model, utilizing illumination conditions relating to the image, can 
transform the image into an internal representation. When the image is subsequently displayed 
on a display screen or printer, the color appearance can be reconstructed from the internal 
representation based on information relating to the output display illumination conditions to 
transform the internal representation to the correct representation for output display. 

[0004] Color appearance models can be used to direct image processing operations to 
supply accurate and appropriate color reproduction, but only in a system with a suitable level of 
color constancy. Several color appearance models have been proposed that suitably compensate 
for a range of illuminants. However, these models cannot be used in digital photography and 
other applications since the color of the scene illumination must be known a priori. Typically 
this information is unavailable. 
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[0005] A digital camera must compensate for illuminants either by measuring the color of 
the scene illumination or estimate the illuminator color from the image data. In a digital imaging 
system, usage of an illumination sensor is impractical 

[0006] One of the problems with color appearance models is that the models are typically 
based on standard color or spectral definitions, such as Munsell color chips, that do not suitably 
cover the gamut of color and illuminance for a particular application. For example, the colors 
and image characteristics of digital cameras can be very different from characteristics in other 
areas. 

SUMMARY OF THE INVENTION 

[0007] What is needed is a technique for achieving color constancy from the image data 
alone. What is further needed is a technique and system that utilize a database that stores actual 
image data and information relating to the image data to base an illuminant detection-training 
algorithm. 

[0008] In accordance with aspects of the present invention, a digital imaging system 
includes a color appearance handling capability that utilizes a database including actual image 
data and information relating to the actual image data to control illumination and color in digital 
images. 

[0009] In one embodiment, the digital imaging system controls illumination and color with 
a color-by-coixelation technology that includes an illuminant detection training algorithm that 
bases control of illumination and color on a database storing actual illuminants, scenes, and 
subjects from representative image acquisition devices such as digital cameras and scanners. In 
one example, illuminant detection training is accurately rendered for a variety of illuminants, 
scenes, and subjects that are expected to be of interest to users of a particular camera or group of 
cameras. In the example, the database is formed by user-group submissions from users of a 
particular camera or group of cameras. The digital imaging system accesses a database capable 
of supplying actual image data including illuminants, chromaticities, colors, and the like. The 
database also stores information relating to the actual image data including tags that identify the 
image acquisition device that acquires a particular image, and information concerning 
manufacturer, model and version of the image acquisition device, and image acquisition devices 
settings at the time of acquisition. 
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[0010] In accordance with another aspect of the present invention, a digital imaging 
system accesses a database capable of supplying actual image data including illuminants, 
chromaticities, colors, and the like. The database further includes information relating to the 
actual image data including tags that identify the image acquisition device that acquires a 
particular image, and information concerning the manufacturer, model and version of the image 
acquisition device, and image acquisition devices settings at the time of acquisition. The digital 
imaging system uses the information for design and improvement of image acquisition devices 
such as digital cameras and scanners. The database includes tags such as manufacturer and 
model tags to screen the digital images stored in the database according to manufacturer and 
model. The database also stores additional private image tag information including 
manufacturer and model information tags, and image data tags including focal length, shutter 
speed, aperture, and the like. The image acquisition device generates tags to derive additional 
information when images created by the devices are acquired and stored in the database. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[001 1] The features of the described embodiments believed to be novel are specifically set 
forth in the appended claims. However, embodiments of the invention relating to both structure 
and method of operation, may best be understood by referring to the following description and 
accompanying drawings. 

[0012] FIGURE 1 is a schematic flowchart illustrating a color appearance model in 
accordance with an aspect of the present invention. 

[0013] FIGURE 2 is a schematic block diagram that shows a digital imaging system with 

a correlation matrix memory for illuminant estimation. 

[0014] FIGURE 3 is a correlation matrix that is configured to correlate data from any 
picture image to reference images under a range of illuminants. 

[0015] FIGURE 4 is a schematic diagram that illustrates a result of vectorization of an 
acquired digital image. 
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[001 6] FIGURE 5 is a schematic block diagram that illustrates an example of an image 
acquisition unit, an electronic imaging device that is suitable for supplying image data to the 
digital imaging system described in FIGURE 1. 

[0017] FIGURE 6 is a schematic block diagram that illustrates another example of an 
electronic acquisition unit, an electronic camera also suitable for supplying image data to the 
digital imaging system described in FIGURE 1. 

DESCRIPTION OF THE EMBODIMENTS S) 

[0018] Referring to FIGURE 1, a schematic flowchart illustrating a color appearance 
model 100 that can be used to convert from RBG tristimulus values to attributes perceptible to 
the human psychovisual system. Similarly, the perceptual attributes can be converted back to 
tristimulus values. The digital imaging system 100 can control processing and display of image 
data to account for both the input and output viewing conditions to store and display color 
characteristics independent of device and viewing condition. Flexible color handling enables a 
range of algorithms and processing for overall color processing. RGB tristimulus input values 
110 are applied to the color appearance model 100 and converted to perceptual attributes 120 
using a forward color appearance model 112, and transformed back to tristimulus output values 
160 using an inverse color appearance model 150. 

[0019] The forward color appearance model 112 includes operations of white point 
estimation 114 and viewing condition estimation 116 using stored image data and information 
including scene attributes 118 and viewing condition tags 119. 

[0020] In an illustrative embodiment, the forward color appearance model 112 and inverse 
color appearance model 150 include a color appearance handling capability that utilizes a 
database of scene attributes 118 and viewing condition tags 119 including actual image data. 
The database further stores information relating to the actual image data to control illumination 
and color in digital images. 

[0021] The perceptual attributes 120 can be modified into processed perceptual attributes 
148 using various techniques including color and processing algorithms that operate on the 
perceptual attributes 120. The various processing techniques applied to uniform color spaces or 
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perceptual attributes 120 include dynamic range compensation 122, color preference adjustments 
124, editing of color properties 126, gamut visualization and mapping 128, compression and 
decompression 130, color quantization 132, and other operations. 

[0022] The inverse color appearance model 150 processes the perceptual attributes using 
stored information and image data including scene attributes 152 and viewing condition tags 
154. 

[0023] Input and output viewing conditions can be set using various techniques. In some 
applications, viewing conditions are estimated by direct measurement, typically for hardcopy or 
soflcopy viewing. Alternatively, correlation techniques or psychophysic operations can be used 
to determine viewing conditions. In other systems, viewing condition tags can be used to select 
from a set of standard viewing conditions. 

[0024] In one embodiment, the forward color appearance model 112 and inverse color 
appearance model 150 control illumination and color with a color-by-correlation technology. 
The color-by-correlation method includes an illuminant detection training algorithm that bases 
control of illumination and color on a database including scene attributes 118 and viewing 
condition tags 119 that stores actual illuminants, scenes, and subjects from representative image 
acquisition devices such as digital cameras and scanners. In one example, illuminant detection 
training is accurately rendered for a variety of illuminants, scenes, and subjects that are expected 
to be of interest to users of a particular camera or group of cameras. Illuminant detection 
training classifies lighting according to an illuminant type such as day lighting, night lighting, 
halogen lighting, fluorescent lighting, and other classifications plus combinations of 
classifications, and modifies processing of images based on the classified illuminant. 

[0025] In the example, the scene attributes 118 and viewing condition tags 119 database is 
formed by user-group submissions from users of a particular camera or group of cameras. The 
digital imaging system accesses a database capable of supplying actual image data including 
illuminants, chromaticities, colors, and the like. The scene attributes 118 and viewing condition 
tags 119 database also stores information relating to the actual image data including tags that 
identify the image acquisition device that acquires a particular image, and information 
concerning the manufacturer, model and version of the image acquisition device, and image 
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acquisition devices settings at the time of acquisition. The database may be a particular 
photography or image database or a variety of photography collections such as photo websites. 

[0026] Referring to FIGURE 2, a schematic block diagram shows a digital imaging 
system 200 with a correlation matrix memory 202 for illuminant estimation. The digital imaging 
system 200 includes an image acquisition unit 210 that is capable of acquiring an image for 
subsequent processing and storage, an illuminant estimator 212, a color correction unit 214, and 
an image output device 216.The image acquisition unit 210 is typically a scanner or a digital 
camera, either a still-image camera or a video camera, but in other embodiments broadly 
includes any other systems for acquiring or capturing images such as computer image 
generators, and the like. For example, an acquired image can be a computer-generated image 
formed using one or more computer-simulated light sources. 

[0027] The illuminant estimator 212 receives acquired images from the image acquisition 
unit 210 and processes the images to estimate the number and type of illuminants associated 
with the acquired image. Illumination characteristics of images are highly variable and a priori 
unknown to the digital imaging system 200. 

[0028] The illuminant estimator 212 estimates illuminant characteristics including number 
and type, and passes a description of the illuminant characteristics and the acquired image to a 
color correction unit 214. The color correction unit 214 processes the acquired image using 
information in the illumination characteristics to correct the color of the acquired image to 
produce a corrected image. The color correction unit 214 supplies the corrected image to the 
image output device 216 for display, printing, storage, transmission, or the like. 

[0029] In various examples, the digital imaging system 200 can have various 
configurations such as a single, integrated unit or a composite system formed from discrete 
components, either independent or in combination. 

[0030] The digital imaging system 200 further includes a processor 218 that is capable of 
executing operations in conjunction with the image acquisition unit 210, the illuminant estimator 
212, and the color correction unit 214 to acquire, analyze, and correct image data. The processor 
218 is typically a programmable device such as a microprocessor, computer, or digital signal 
processor, but may otherwise be a computational unit implemented in a programmable logic 
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array, microcontroller, or the like. Although FIGURE 2 shows processor 218 as being internal 
to the digital imaging system 200, in various examples the processor 218 can be internal or 
external with respect to the digital imaging system 200. In one example, the processor 218 can 
be a host computer that is external to other components of the digital imaging system 200. 

[0031] Components of the digital imaging system 200 are described in terms of the 
functions performed rather than as a particular structural form. Any structures capable of 
performing the described functions as known by those having ordinary skill in the art may be 
used to perform the functions. Typically, the described functions can be performed according to 
control of a processor of any type. For example, the illuminant estimator 212 may be a 
microprocessor, an application specific integrated circuit (ASIC), a discrete circuit, a state 
machine, or any other circuit or component that is known in the art by one of ordinary skill in 
the art and capable of performing the described functions. 

[0032] The digital imaging system 200 analyzes the acquired image to determine digital 
adjustments to the image that can be made by the color correction unit 214 to correct for 
characteristics of the light source used to illuminate the image. The digital imaging system 200 
performs mathematical manipulation of image data using a technique for attaining color 
constancy called "color by correlation" to solve for a white-point in images. The color-by- 
correlation technique improves color digital imaging quality by quickly, accurately, and robustly 
estimating illuminant information associated with a color digital image by correcting image 
color based on the illuminant. The color-by-correlation technique exploits the correlation that 
exists between image colors and scene illuminants. For example, the most red camera 
measurement can only occur under red light having the highest proportion of red wavelength. 
Accordingly, the camera measurement with the highest red reading correlates strongly with the 
most red light. All camera measurements correlate to a greater or lesser degree with different 
light colors. The illuminant estimator 212 measures the correlation between all image colors and 
all lights to form a very accurate estimate of the color of the scene illuminant. 

[0033] The color-by-correlation technique is insensitive to aperture colors and, for images 
with few illuminant colors, permits estimation such as determination of a maximum likelihood 
estimator. 
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[0034] The illuminant estimator 212 uses the correlation matrix memory 202, which may 
also be termed an "associative matrix memory," to correlate image colors and lights using 
Bayesian statistical analysis. The illuminant estimator 212 builds the correlation matrix memory 
202 to correlate data from any image to the set of possible scene illuminants. The illuminant 
estimator 212 builds the correlation matrix memory 202 by first forming a set of reference 
surfaces that are illuminated by a particular source. The set of reference surfaces are plotted in 
chromaticity space. The illuminant estimator 212 then takes a convex hull of the chromaticities 
of the reference surfaces. The illuminant estimator 212 then rearranges the chromaticities to 
obtain reference ranges for particular source illuminant. For example, the matrix may be 
configured so that the presence of a chromaticity in a reference range is designated a "1" and the 
absence of a chromaticity in a range is designated as a "0" entry. 

[0035] Columns of the matrix are a rearrangement of a two-dimensional chromaticity 
space into a list of binary points. For a particular color formed under a particular illuminant, a 
point called a chromaticity coordinate is set to "1" if and only if that color can occur under that 
illuminant. For example, the reddest chromaticity can only occur under the reddest illumination. 
Rows of the matrix correspond to a list of all plausible illuminants detected by a device. 

[0036] The digital imaging system 200 computes data for the matrix using a set of 
reference surface colors, for example a color chart or a set of standard surfaces. According to 
one illustrative technique, one or more databases 220 of real- world images are used to accurately 
render for a variety of illuminants, scenes, and subjects. The various real-world images can be 
tagged according to information relating to specific imaging devices such as particular digital 
camera models or even a particular digital camera. Tags in the database 220 contain various 
types of information. The system renders the images more accurately by exploiting the 
information contained in the tags. 

[0037] For each illuminant, the digital imaging system 200 computes chromaticities of the 
reference set for the column corresponding to the illuminant, first by multiplying the illuminant 
by reflectances stored for the reference surface, second by calculating chromaticities as detected 
by the image acquisition unit 210. The chromaticities correspond to the reference colors under 
the illuminant. 
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[0038] The reference range for the particular illuminant is a polygon derived from taking 
the convex hull of the points. In the column corresponding to the particular illuminant, the 
digital imaging system 200 activates the chromaticities of the points within the reference range 
to a "1" value, and deactivates other points to a "0" value. 

[0039] Referring to FIGURE 3, a correlation matrix 300 is configured to correlate data 
from any picture image to reference images under a range of illuminants. Columns, the vertical 
dimension of the matrix memory, are rearrangements of the two dimensional chromaticity space 
into a raster list of points. The correlation matrix 300 has a first column that corresponds to a 
shaded polygon, which contains the chromaticities of the reference range under illuminant T. 
The digital imaging system 200 repeats the procedure for each column corresponding to all 
illuminants. The number of illuminants can be limited to attain only a desired result precision. 
The illuminant may be selected from among a group of sources. In an illustrative embodiment, 
the illuminant may be selected from among sources that correspond to the particular model of 
image acquisition unit 210. 

[0040] Rows of the correlation matrix 300 correspond to a raster list of chromaticities for 
all possible luminants as detected by the device. The illustrative system facilitates computation 
of data for the correlation matrix by utilizing a set of reference surface colors such as a color 
chart or standard surface set. For each illuminant and associated column , the system computes 
chromaticities of the reference set. The system multiplies the illuminant by the reference surface 
reflectances and calculates the chromaticities detected by the imaging device. In the column 
corresponding to the particular illuminant, the chromaticities of the points within the reference 
range are activated by assigning a value "1" in the matrix entry. Other entries are set to value 
"0". The system repeats the procedure for each column corresponding to all illuminants. 

[0041] The results of generating the correlation matrix memory 202 depend on the spectral 
sensitivities of the detector, the reference surfaces, and the illuminants, all of which are known 
for a reference image. The digital imaging system 200 can perform illumination detection 
training as part of calibration of the digital imaging system 200 without assuming either device 
linearity or spectral basis. 

[0042] After generating the correlation matrix memory 202, the digital imaging system 
200 can plot image data in the same chromaticity space with a list of chromaticities created in 
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vector form. Referring to FIGURE 4, a schematic diagram illustrates a result of vectorization of 
an acquired digital image 410. 

[0043] The image acquisition unit 210 such as a camera, scanner, or the like generates a 
digital image picture by converting RGB color data to chromaticities and creating an image 
vector 412 corresponding to the values existing in the scene. The image vector is similar to the 
columns in the correlation matrix but contains "1" values in the positions of chromaticities that 
appear in the image and contains "0" values for chromaticities that do not appear in the image. 

[0044] The illuminant estimator 212 applies the correlation matrix stored in the correlation 
matrix memory 202 to an image vector formed by the image acquisition unit 210. The illuminant 
estimator 212 multiplies the image vector by each column in the correlation matrix to create a 
new matrix. In the new matrix, every row that represents a chromaticity that did not exist in the 
picture image is assigned a value of "0" and the rows representing chromaticities that did exist in 
the picture image are assigned to either "0" or "1". In this manner, the rows are either 
inactivated or allowed to be activated, as in the correlation matrix, depending on the existence of 
the color in the picture image. The illuminant estimator 212 multiplies the image vector v with 
each column in the correlation matrix M to produce a sum vector s according to equation (1): 

s = v(t) • M 
where v(t) is the transpose of v. 

[0045] In computation of the sum vector, rows are activated or deactivated depending 
upon whether a particular color is present in the image. Accordingly, multiplication of the 
image vector by rows in the correlation matrix activates rows that exist in the image and 
deactivate rows corresponding to non-image chromaticities. The sum vector is a cumulative 
correlation of values that represents the number of image chromaticities that are consistent with 
a particular illuminant. Specifically, the first entry of the sum vector specifies the number of 
input colors that are consistent with a first illuminant. The second entry of the sum vector 
specifies the number of input colors that are consistent with a second illuminant, in sequence for 
all illuminants. The entry with the largest number of consistent input numbers designates the 
appropriate illuminant. 
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[0046] Summing of the columns to form the sum vector results in a vector representing 
likelihood that each reference source is an illuminant of the scene. Values in the sum vector can 
be plotted on a density plot in a chromaticity space in which each value is plotted at the 
chromaticity of the illumination for that particular column. Statistical methods can be used on 
the plot to estimate the likely illumination of the scene. 

[0047] The color correction unit 214 performs a threshold operation to convert the sum 
vector to a binary vector. In one example, a threshold can be selected that is equal to the number 
of input color, typically producing a result that only a single element of a binary vector is 
selected, corresponding to an illuminant that is consistent with all input colors. In other 
examples, a threshold that is less than the total number of input colors can be selected since 
aperture colors are not generally consistent with other image colors. The binary vector 
designates the illuminant that is most consistent with the input colors. 

[0048] The correlation matrix Mhas elements set to "1" and can be analyzed as a special 
case of the Bayesian statistical model in which all colors under a particular illuminant, as well as 
all illuminants, are equally likely. An element in the correlation matrix is set to "1" if the image 
chromaticity of the row of the element is consistent with the illuminant of the column of the 
element. 

[0049] Information, such as image database information, can be used to determine the 
probability that a chromaticity for a particular row occurs under the illuminant of a particular 
column. The information may be determined experimentally or from a theoretically determined 
distribution. Baye's rule permits calculation of the probability of an illuminant at a column 
given the fact that a chromaticity appears in an image. Assuming independence of surface 
reflectances that could appear in a scene, the probability of an illuminant j given image 
chromaticities / and a second image chromaticity k 9 is proportional to p(j\I) pQ\k). Equation 
p(j\I)pQ\k) can be written as logipQW) + log(p(j\k)). 

[0050] By initializing the position ij in the correlation matrix to the value log(p(/|7)), the 
correlation matrix can be used to determine the most probable estimate of white. The maximum 
value sum vector corresponds to the most likely luminant. 
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[0051] Referring to FIGURE 5, a schematic block diagram illustrates an electronic camera 
502 that is suitable for usage as an image acquisition unit 210. A sensor such as a charge- 
coupled device (CCD) photo sensor 504 that detects light passing through a lens system 506 to 
form an image. The alternative illustrative zoom and focus control system 500 has a lens system 
506 that also includes a plurality of lens groups, each of which includes one or more lenses. The 
present example also has three lens groups, first lens group 508, second lens group 510, and 
third lens group 512. The first lens group 508 is a fixed-objective lens. The second lens group 
510 is a moving group lens that contains an aperture stop (not shown). The second lens group 
510 moves in conjunction with the third lens group 512 to cause magnification to change when 
the lens system 506 moves. The second lens group 510 moves in a nonlinear manner to hold the 
image plane in a relatively constant position as the lens system 506 is zoomed. The second lens 
group 510 is also moved to focus the lens system 506 after reaching a particular focal length. 
The third lens group 512 is also a moving group of lenses in the lens system 506. The third lens 
group 512 is also a moving lens group. The third lens group 512 moves in a linear manner to 
change the focal length of the camera as the lens zooms. 

[0052] The CCD photo sensor 504 is a two-dimensional array of charge-coupled photo 
sensors that captures an image focused onto the photo sensors by the lens system 506. 
Individual photo sensor sites are pixels with an associated color such as red, green, or blue. 
CCD photo sensor 504 is exposed to light and charge integrates at the individual pixel site 
proportional to the number of photons received at the site. Clock drivers 514 are connected to 
the CCD photo sensor 504 and convey clock signals to control the read-out process of CCD 
photo sensor 504. Image processing hardware (not shown) generates the clock signals and sends 
the clock signals from the image processing hardware through the clock drivers 514 to the CCD 
photo sensor 504. The clock drivers 514 supply clock signals with high current and sufficient 
frequencies to drive highly capacitive CCD control lines. 

[0053] A low pass filter 516 is connected to the CCD photo sensor 504 for anti-alias 
filtering to avoid optical moire effects resulting from the discrete CCD pixel structure. An 
example of the low pass filter 516 includes two birefringent quartz plates (not shown) and a 
quarter-wave plate (not shown). One birefringent quart plate produces filtering with respect to a 
horizontal direction of the CCD array. The other birefringent plate filters in a vertical direction 
of the CCD array. The quarter-wave plate functions is a depolarizer. The term low-pass filter 
indicates imaging of only a low spatial frequency image. 
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[0054] The CCD photo sensor 504 generates image signals and passes the signals through 
an analog to digital converter (ADC) 518 to the image processing hardware. Row-by-row pixel 
image data from the CCD photo sensor 504 are an analog voltage passed to the analog to digital 
converter 518. The analog to digital converter 518 amplifies and digitizes the image signal. The 
digitization process generates an N-bit digital word for each pixel. The analog to digital 
converter 518 clocks pixel data into the image processing hardware. 

[0055] A shutter/aperture module 520 is interposed between the second lens group 508 and 
the third lens group 510. The shutter/aperture module 520 includes a shutter 522, apertures 524, 
and a neutral density filter 526. The shutter 522 is a blade switched into the optical path 528 of 
the lens system 506 to prevent light from reaching the CCD photo sensor 504. The shutter 522 
blocks light at the end of an exposure time to complete image capture and when the camera is 
powered off to protect the CCD photo sensor 504 from receiving excessive light and damaging 
individual sensor elements. 

[0056] The apertures 524 are multiple blades, each containing a hole. Different blades 
typically have different diameter holes. Different apertures can be switched into the optical path 
528 to reduce the light transmitted through the lens system 506 to the CCD photo sensor 504. 
Different apertures 524 are used to control light exposure and focus depth of field. A typical 
still-image electronic camera has one or two aperture blades. In other designs, the aperture 524 
may be composed of a diaphragm with continuously variable aperture hole sizes to supply a 
greater number of selectable apertures. 

[0057] The neutral density filter 526 is an additional blade that can be switched into the 

optical path 528. The neutral density filter 526 also reduces the light passed through the lens 
system 506 to the CCD photo sensor 504. The aperture 524 and the neutral density filter 526 
function similarly, but differ in that the neutral density filter 526 can be used to reduce the 
amount of light passing through the lens system 506 without affecting the focus depth of field. 
In contrast, usage of an aperture to reduce light to the CCD photo sensor 504 changes the depth 
of focus. The neutral density filter 526 can be used in conjunction with the apertures 524 to 
further reduce the level of light passing to the CCD photo sensor 504. 

[0058] The shutter/aperture module 520 is controlled by signals passed from a camera 
control element (not shown) via solenoids 530. Individual blades of the shutter 522, the 
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apertures 524, and the neutral density filter 526 are actuated in and out of the lens optical path 
528 by a plurality of solenoids including a shutter solenoid 529, an aperture solenoid 530, and a 
neutral density filter solenoid 535. The solenoids are driven by coil drivers 532 which supply 
voltage and current for actuating the blades into position in a timely manner. The coil drivers 
532 are controlled by signals from a processor (not shown) such as a central processing unit 
(CPU), a microcontroller, or control logic. The camera control element may be software or 
firmware that executes on the processor. 

[0059] The processor determines relative positioning of the second lens group 510, and the 
third lens group 512, thus controlling zoom and focus functionality. The processor executes 
application programs that supply control information to a motor driver 534, an executing 
program code supplying control signals to a first stepper motor 538 and a second stepper motor 
540. The stepper motors 538 and 540 physically control the position of the second lens group 
510 and the third lens group 512, respectively. 

[0060] In the illustrative electronic camera 502, the motor driver 534 sends signals to the 
stepper motor 538 that is connected to the second lens group 510 by gear reduction 550 and a 
lead screw 552. Similarly, the motor driver 534 sends signals to the stepper motor 538 that is 
connected to the third lens group 512 also by gear reduction 550 and a lead screw 552. The 
stepper motors 538 and 540 receive signals from the processor via the motor driver 534 that 
determine the position and motion of the second lens group 510 and the third lens group 512. In 
alternative systems, other types of motors and drive mechanisms can be used to controls lens 
position adjustment of the second and third lens groups 510 and 512. Photo sensors 554 are 
connected to the motor and drive mechanism for the second and third lens groups 510 and 512 to 
monitor positioning of the lens groups 510 and 512. The processor determines the initial 
positions of the second and third lens groups 510 and 512 by moving the lens groups toward the 
photo sensors 554 and detecting when flag 556 mounted on the second and third lens groups 510 
and 512 reaches the photo sensors 554. The position at which the photo sensors 554 detect the 
flags is a home position. The processor measures the positions of the second and third lens 
groups 510 and 512 relative to the home position. The processor tracks the number of steps the 
stepper motors 538 and 540 execute in all moves relative to the home position. 
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[0061] The motor drivers 534 supply voltage and current to the stepper motors 538 and 
540, thus determining the position and motion of the second lens group 510 and the third lens 
group 512. The motor drivers 534 are controlled by signals from the processor. 

[0062] Referring to FIGURE 6, a schematic block diagram illustrates another electronic 
camera 602 that is suitable for usage as an image acquisition unit 210.. An image is detected at a 
sensor, for example a charge-coupled device (CCD) photo sensor 604 that detects light passing 
through a lens system 606. In the illustrative zoom and focus control system 600, the lens 
system 606 includes a plurality of lens groups, each of which includes one or more lenses. One 
example has three lens groups, first lens group 608, second lens group 61 0, and third lens group 
612, The second lens group 61 0, termed a variator, changes the effective focal length of the lens 
and moves in a linear manner. The first lens group 608 moves in a nonlinear manner relative to 
the linear motion of the second lens group 610 and functions as a compensator. The first lens 
group 608 functions to hold the image plane relatively constant as the lens is zoomed over the 
range of focal lengths of the lens system 606. The third lens group 612 is a positive is a positive 
element that is moved to focus the lens system 606. 

[0063] The CCD photo sensor 604 is a two-dimensional array of charge-coupled photo 
sensors used to capture the image that is focused onto the photo sensors by the lens system 606, 
The individual photo sensor sites are defined as pixels and have an associated color such as red, 
green, or blue. As the CCD photo sensor 604 is exposed to light, charge integrates at the 
individual pixel site proportional to the number of photons received at the site. Clock drivers 
614 are connected to the CCD photo sensor 604 and propagate clock signals that are used to 
control the read-out process of the CCD photo sensor 604. Image processing hardware (not 
shown) generates the clock signals. The clock signals propagate from the image processing 
hardware through the clock drivers 614 to the CCD photo sensor 604. The clock drivers 614 
supply clock signals with high levels of current at sufficient frequencies to drive highly 
capacitive CCD control lines. 

[0064] A low pass filter 616 is connected to the CCD photo sensor 604 for usage as an 
anti-aliasing filter to avoid optical moire effects that occur due to the discrete nature of the CCD 
pixel structure. One suitable example of the low pass filter 616 includes two birefrtngent quartz 
plates (not shown) and a quarter-wave plate (not shown). One of the birefringent quart plates 
produces filtering with respect to a horizontal direction of the CCD array. The second 
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birefiingent plate produces filtering in a vertical direction of the CCD array, 90° shifted from the 
horizontal direction. The quarter-wave plate functions as a depolarizer. The term low-pass filter 
indicates imaging of only a low spatial frequency image. 

[0065] The CCD photo sensor 604 generates image signals that are passed through an 
analog to digital converter (ADC) 618 to the image processing hardware. Row-by-row pixel 
image data is read from the CCD photo sensor 604 as an analog voltage and is passed to the 
analog to digital converter 618. The analog to digital converter 618 amplifies and digitizes the 
image signal. The digitization process generates an N-bit digital word for each pixel. The 
analog to digital converter 618 clocks the digital words for the pixels into the image processing 
hardware. 

[0066] A shutter/aperture module 620 is interposed between the first lens group 608 and 
the second lens group 610. The shutter/aperture module 620 includes a shutter 622, apertures 
624, and a neutral density filter 626. The shutter 622 is a blade that is switched into the optical 
path 628 of the lens system 606 to prevent light from reaching the CCD photo sensor 604. The 
shutter 622 typically is controlled to block the light at the end of an exposure time to complete 
image capture. The shutter 622 is also closed when the camera is powered off to protect the 
CCD photo sensor 604 from receiving excessive light, potentially causing damage to the 
individual sensor elements. 

|0067] The apertures 624 are multiple blades containing different diameter holes. An 
aperture blade can be switched into the optical path 628 to reduce the amount of light 
transmitted through the lens system 606 to the CCD photo sensor 604. Different apertures 624 
are used to control light exposure and to control the focus depth of field. A typical electronic 
camera that is used for still image reception has one or two aperture blades. Alternatively, the 
aperture 624 may be composed of a diaphragm with continuously variable aperture hole sizes to 
supply a greater number of selectable apertures, 

[0068] The neutral density filter 626 is an additional blade that can be switched into the 
optical path 628. The neutral density filter 626 also reduces the amount of light that is 
transmitted through the lens system 606 to the CCD photo sensor 604. Although an aperture 624 
and the neutral density filter 626 are similar in function, the neutral density filter 626 can be 
used to reduce the amount of light passing through the lens system 606 without affecting the 
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focus depth of field. Usage of an aperture to reduce light to the CCD photo sensor 604 always 
affects the depth of focus. The neutral density filter 626 can be used in conjunction with the 
apertures 624 to further reduce the level of light passing to the CCD photo sensor 604. 

|0069] The shutter/aperture module 620 is controlled by signals passed from a camera 
control block (not shown) via solenoids (not shown). Individual blades of the shutter 622, the 
apertures 624, and the neutral density filter 626 are actuated in and out of the lens optical path 
628 by a solenoid. The individual solenoids are driven by a solenoid driver (not shown) which 
supplies the voltage and current for actuating the blades into position in a timely manner. The 
solenoid drivers are controlled by signals from a processor 670 such as a central processing unit 
(CPU), a microcontroller, or control logic. The camera control block may be software or 
firmware that executes on the processor 670. 

1 0070] The processor 670 determines relative positioning of the first lens group 608, the 
second lens group 610, and the third lens group 612, thus controlling zoom and focus 
functionality. The processor 670 executes application programs that supply control information 
to a motor driver 634, an executing program code supplying control signals to a DC motor 636 
and a stepper motor 638. The DC motor 636 physically controls the position of the first lens 
group 608 and the second lens group 610 of the lens system 606. The stepper motor 638 
physically controls the position of the third lens group 612. The first lens group 608 and the 
second lens group 610 are held by a lens barrel 640. DC motor 636 is connected to the lens 
barrel 640 and drives the rotation of the lens barrel 640 via a set of gears (not shown) between 
the lens barrel 640 and the DC motor 636. As the lens barrel 640 rotates, the positions of the 
first lens group 608 and the second lens group 610 are adjusted through the operation of cam 
slots (not shown) inside the lens barrel 640. A lens cam switch 642 is mounted on the lens 
barrel 640 and sends signal transitions to the processor 670 as the lens barrel 640 rotates through 
a plurality of zoom positions. In one example, the electronic camera 602 has three zoom 
positions of wide, tele, and retract. A slider potentiometer 644 is connected to the lens barrel 
640. As the lens barrel 640 rotates between the wide and tele zoom positions, the slider 
potentiometer 644 produces a fine zoom position information. A cam slot 646 in the lens barrel 
640 drives the slider potentiometer 644 depending on the position of the wide and tele zoom 
position. The processor 670 determines the fine zoom position by reading the voltage obtained 
from the center tap of the slider potentiometer 644 via an analog-to-digital converter (ADC) 648. 
Fine zoom position values produced by the slider potentiometer 644 are calibrated by recording 
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the slider potentiometer values when the lens cam switch 642 is positioned at the tele and wide 
positions. 

[0071] In the illustrative electronic camera 602, the motor driver 634 sends signals to the 
stepper motor 638 that is connected to the third lens group 612 by gear reduction 650 and a lead 
screw 652. The stepper motor 638 receives signals from the processor 670 via the motor driver 
634 that determine the position and motion of the third lens group 612. In alternative systems, 
other types of motors and drive mechanisms can be used to controls lens position adjustment of 
the third lens group 612. Photo sensors 654 are connected to the motor and drive mechanism for 
the third lens group 612 to monitor positioning of the third lens group 612. The processor 670 
determines the initial position of the third lens group 612 by moving the lens group toward the 
photo sensor 654 and detecting when flag 662 mounted on the third lens group 612 reaches the 
photo sensor 654. The position at which the photo sensor 654 detects the flag is a home 
position. The processor 670 measures the position of the third lens group 612 relative to the 
home position. The processor 670 tracks the number of steps the stepper motor 638 executes in 
all moves relative to the home position. 

[0072] The motor driver 634 supplies voltage and current to the DC motor 636 and the 
stepper motor 638, thus determining the position and motion of the first lens group 608 and 
second lens group 610, and the third lens group 612. The motor driver 634 is controlled by 
signals from the processor 670. 

[0073] A temperature sensor 660 inside the electronic camera 602 measures temperature 
and is connected to the processor 670. The processor 670 includes a processing logic that is 
capable of adjusting a starting position of the lenses to adjust focus for temperature differences. 

[0074] In accordance with another aspect of the invention, the digital imaging system 200 
can access a database such as the scene attributes 118 and viewing condition tags 119 database 
described with reference to FIGURE 1 that is capable of supplying actual image data including 
illuminants, chromaticities, colors, and the like. The scene attributes 118 and viewing condition 
tags 119 database further includes information relating to the actual image data including tags 
that identify the image acquisition device that acquires a particular image, and information 
relating to manufacturer, model and version of the image acquisition device, and image 
acquisition devices settings at the time of acquisition. The digital imaging system 200 uses the 
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information for design and improvement of image acquisition devices such as digital cameras 
and scanners. The database includes tags such as manufacturer and model tags to screen the 
digital images stored in the database according to manufacturer and model. The database also 
stores additional private image tag information according to tags that are generated by the image 
acquisition devices to derive additional information when images created by the devices are 
acquired and stored in the database. Examples of tag information include identification of 
manufacturer and model. Other tag information relates to imaging characteristics such as focal 
length, shutter speed, aperture, and the like. 

[0075] The digital imaging system 200 can use the tag information to perform statistical 
analysis on the usage of the image acquisition unit 110. For example, tags can be used to signify 
whether a camera is more often used to take photographs in a landscape configuration or a 
portrait configuration. A statistical analysis can be performed to determine whether a 
preponderance of images is taken in a particular configuration. As a result of the statistical 
analysis, camera designers may arrange imaging components or photodetectors in a particular 
manner to exploit information relating to the most common usage of the camera. 

[0076] The digital imaging system 100 can use the database to perform various types of 
statistical analysis. In one aspect the digital imaging system 100 can perform an analysis that 
determines the type of images that are acquired, classifying the images according to subject 
matter, illuminant, color content, scene, whether images are more commonly acquired of people 
or natural scenes, action images or posed images, depth of field of the image content of 
particular interest, and many other considerations. 

[0077] The digital imaging system 100 can analyze the database to determine percentages 
of particular image characteristics. For example, the system may perform illuminant analysis to 
determine the percentage of images taken with outdoor light, fluorescent lighting, halogen 
lighting, other indoor lighting, night lighting, flash pictures, and combinations of lighting. The 
various types of images may be analyzed to improve illuminant detection algorithms. 

[0078] Similarly, the digital imaging system 100 can analyze images in the database to 
improve other algorithms and techniques including geometric compression, auto-sharpening, 
red-eye removal, and the like. The digital imaging system 100 can also use the database to 
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develop artificial intelligence analysis functions such as scene recognition, face recognition, and 
the like. 

[0079] The digital imaging system 100 can use the tagged information in the manner of a 
flight recorder to determine common methods of transferring images such as facsimile transfer, 
copying, printing, and the like. 

[0080] Tags can be used to collect information concerning the manner of usage of the 
image acquisition unit 110, particularly concerning the usage of the image acquisition unit as an 
entry node to a network. For example, images may be transferred to a network using a camera 
as an input node, or images may be transferred through a personal computer, a photography 
shop, a laboratory, or the like. Cameras may be improved to simplify or otherwise facilitate the 
transfer of images to the internet. 

[0081] Various features of the image acquisition unit 110 may be tagged to determine the 
usage of particular device features. Users may commonly and frequently use particular camera 
features and may avoid usage of other features. The digital imaging system 100 can analyze 
usage information to determine features that are not used sufficiently to justify the cost of 
including the feature in the imaging device. Image device developers can use the statistical 
information to eliminate or modify the infrequently used features based on the information. 

[0082] While the invention has been described with reference to various embodiments, it 
will be understood that these embodiments are illustrative and that the scope of the invention is 
not limited to them. Many variations, modifications, additions and improvements of the 
embodiments described are possible. For example, those having ordinary skill in the art will 
readily implement the steps necessary to provide the structures and methods disclosed herein, and 
will understand that the process parameters, materials, and dimensions are given by way of example 
only and can be varied to achieve the desired structure as well as modifications which are within the 
scope of the invention. Variations and modifications of the embodiments disclosed herein may be 
made based on the description set forth herein, without departing from the scope and spirit of the 
invention as set forth in the following claims. For example, one or ordinary skill in the art could 
similarly apply the first and second quality-of-service techniques to the other interconnect 
structures described herein. 
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In the claims, unless otherwise indicated the article "a" is to refer to "one or more than 
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